Ranging apparatus, ranging method and imaging system

ABSTRACT

A ranging apparatus has an imaging unit including a pixel group for acquiring first and second images formed by luminous fluxes having passed through first and second pupil areas of an imaging optical system, and a calculation unit configured to create third and fourth images by performing convolution integrals on the first and second images with corrected first and second image modification functions, and to calculate a distance up to the subject by comparison of the third and fourth images, wherein the corrected first and second image modification functions are formed by causing centroid positions calculated based on data of sampling points of the first and second image modification functions corresponding to pixel arrangement of the pixel group to each coincide with a sampling point closest to the centroid position; and the convolution integral is performed by taking the sampling point closest to the centroid position as a reference point.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a ranging apparatus, a ranging methodand an imaging system, more particularly to a ranging apparatus and aranging method which are used for an imaging system such as a digitalstill camera, a digital video camera, or the like.

2. Description of the Related Art

There are known distance detection techniques used for digital stillcameras and video cameras. In Japanese Patent Application Laid-Open No.2002-314062, a solid-state imaging device having a ranging function in aportion of pixels thereof, which is configured to detect a distancethrough a phase difference method, is proposed. This phase differencemethod includes processes of estimating a gap amount between opticalimages (respectively called as an A image and a B image, and also calledas AB images collectively) produced by luminous fluxes having passedthough different areas on a pupil of a camera lens and of calculating adefocus amount using triangulation with stereo images; ranging isthereby performed. According to this method, since any lens is notrequired to be moved for distance measurement, high-accurate andhigh-speed ranging is enabled unlike a conventional contrast method.Real-time ranging is also enabled when moving images are taken.

If vignetting in a luminous flux is caused by the frame of a taking lensor the like, the A and B images become different to each other, whichcauses the accuracy of estimating the image gap amount to be reduced andalso the ranging accuracy to be degraded. An image-shape modificationtechnique is disclosed in US 2012/0057043 A1. In this technique, imagemodification filters are formed using line spread functionscorresponding to pupil areas for forming the A and B images. The shapesof the A and B images are modified by performing convolution integral onthem with the image modification filters, respectively, after thefilters have been mutually interchanged. Since the magnitude of theimage modification filter (line spread function) varies depending ondefocus amounts, the shape of the image modification filter is correctedin accordance with a calculated defocus amount, and the processes ofmodifying the images and recalculating the defocus amount are repeatedlyperformed. Desirable image modification filters are formed using adefocus amount close to the right value, which is acquired through suchloop processing of the ranging calculation; thereby, the accuracy ofimage modification and distance measurement can be enhanced.

The AB images in the technique disclosed in US 2012/0057043 A1 arediscrete data which are composed of the values acquired at respectivepixels. In order to perform convolution integral on the respectiveimages, the image modification filters are formed by discretizing therespective continuous line spread functions in accordance witharrangement spacing of pixels, taking the centroid positions of therespective continuous line spread functions as reference points. Thereis, however, a case in which the centroid position calculated fromdiscrete values of the image modification filter differs from thereference point depending on the distance up to a subject (defocusamount). This error is called as a centroid error. If such a centroiderror exists, the centroid position of a modified image deviates fromthat of the original image. The deviation causes an error to arise inestimated values of the image gap amount and the defocus amount. Thiscentroid error is independent with respect to the shape error of theimage modification filter, and even though loop processing of theranging calculation is performed, either the centroid error or the shapeerror remains. The calculated values of the image gap amount and thedefocus amount therefore do not converge, which causes the ranging timeto increase and also causes the ranging accuracy to be degraded.

SUMMARY OF THE INVENTION

The present invention can provide a ranging apparatus, a ranging methodand an imaging system that enable ranging with high accuracy at highspeed.

A ranging apparatus according to the invention includes an imagingoptical system for forming an image of a subject, an imaging unitincluding a pixel group for acquiring first and second images formed byluminous fluxes having passed mainly through first and second pupilareas, respectively, of an emitting pupil of the imaging optical system,and a calculation unit, wherein the calculation unit is configured tocreate a third image by performing convolution integral on the firstimage with a corrected first image modification function as well as tocreate a fourth image by performing convolution integral on the secondimage with a corrected second image modification function, and tocalculate a distance up to the subject by comparison of the third andfourth images; the corrected first and second image modificationfunctions are formed by causing respective centroid positions, which arecalculated based on data of sampling points of respective first andsecond image modification functions corresponding to pixel arrangementof the pixel group, to each coincide with the sampling point closest tothe centroid position; and the convolution integral is performed bytaking the sampling point closest to the centroid position as areference point.

A ranging method of the invention includes using an imaging opticalsystem for forming an image of a subject; acquiring first and secondimages formed by luminous fluxes having passed mainly through first andsecond pupil areas, respectively, of an emitting pupil of the imagingoptical system by use of an imaging unit having a pixel group; andmeasuring a distance up to the subject based on the first and secondimages acquired by the imaging unit. The ranging method has steps offorming first and second image modification filters; performingmodification calculation by creating a third image by performingconvolution integral on the first image with the first imagemodification filter, and also creating a fourth image by performingconvolution integral on the second image with the second imagemodification filter; and calculating the distance up to the subjectbased on an image gap amount between the third and fourth images,wherein the step of forming the first and second image modificationfilters includes a process of performing discretization and centroidcorrection; and the image modification filters are each formed bydiscretizing an image modification function according to a pixelarrangement of the pixel group at the discretizing process to form adiscrete function, and by causing the centroid position of the discretefunction to coincide with a reference point, which is discrete dataclosest to the centroid position, at the process of performing centroidcorrection.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic cross-sectional view of a configuration example ofa ranging apparatus according to first embodiment of the presentinvention.

FIG. 2 is a flow chart explaining an example of the ranging methodaccording to the first embodiment of the present invention.

FIG. 3A is a schematic cross-sectional view explaining a configurationexample of the ranging apparatus according to the first embodiment ofthe present invention.

FIG. 3B is a schematic cross-sectional view explaining a configurationexample of the ranging apparatus according to the first embodiment ofthe present invention.

FIG. 4 is a flow chart of a process for forming an image modificationfilter according to a first embodiment of the present invention.

FIG. 5A illustrates an image modification function according to thefirst embodiment of the present invention.

FIG. 5B illustrates a discrete function according to the firstembodiment of the present invention.

FIG. 5C illustrates a discrete function according to the firstembodiment of the present invention.

FIG. 5D illustrates the image modification filter according to the firstembodiment of the present invention.

FIG. 6 is a graph explaining as to the results of measuring the distanceup to a subject when image modification filters according to the firstembodiment of the present invention have been used.

FIG. 7A illustrates another discrete function according to anotherembodiment of the present invention.

FIG. 7B illustrates another image modification filter according toanother embodiment of the present invention.

FIG. 8 is a flow chart explaining an example of the ranging methodaccording to another embodiment of the present invention.

FIG. 9A is a schematic cross-sectional view explaining a configurationexample of the ranging apparatus according to another embodiment of thepresent invention.

FIG. 9B is a schematic cross-sectional view explaining a configurationexample of the ranging apparatus according to another embodiment of thepresent invention.

FIG. 10 is a schematic cross-sectional view explaining a configurationexample of the ranging apparatus according to another embodiment of thepresent invention.

DESCRIPTION OF THE EMBODIMENTS

Preferred embodiments of the present invention will now be described indetail in accordance with the accompanying drawings.

Configuration examples of a ranging apparatus and a ranging methodembodying the present invention will be described below. A configurationexample of the ranging apparatus 100 according to this embodiment isillustrated in FIG. 1. An imaging optical system 102 forms an image ofan external subject on the surface of an imaging device 103. The imagingdevice 103 is structured of a plurality of pixels. The ranging apparatus100 is provided with a wiring 115 for reading out acquired signals and acalculation unit 120 for calculating the distance up to the subjectusing acquired signals. The calculation unit 120 is constituted by, forexample, a signal processing board including a CPU and a memory. Thedistance detecting device 100 is provided with a recording unit 130 forrecording read-out signals or calculation results.

The ranging method according to this embodiment will be explained alongthe flow chart shown in FIG. 2. At step S1, processing count N is set to0. At step S2, AB images are acquired by means of the ranging apparatus100 shown in the configuration example of FIGS. 3A, 3B. The imaging lens102 in FIG. 3A forms an image of an external subject on the surface ofthe imaging device 103. The imaging device 103 is provided with aplurality of pixels 101 shown in FIG. 3B. The pixels 101 are eachprovided with, for example, a micro lens 107 and photoelectricconversion sections 105, 106 mounted in a board 104. The photoelectricconversion sections 105, 106 are disposed below shading members 108,109, respectively. Although not shown in FIGS. 3A and 3B, the rangingapparatus 100 is provided with the wiring, the recording unit, CPU, andthe like for reading out, recording and processing the acquired signalsshown in FIG. 1.

The distance between the imaging lens 102 and the imaging device 103 islarge with respect to the size of each of the pixels 101. Therefore,luminous fluxes having passed through different areas on the emittingpupil surface of the imaging lens 102 enter the surface of the imagingdevice 103 as luminous fluxes having different incident angles with eachother. At the photoelectric conversion section 105 included in eachpixel of the imaging device 103, a luminous flux is detected that haspassed mainly through an area 112 (a first pupil area), whichcorresponds to a first direction 110, of the emitting pupil (emittingpupil of the optical system forming a subject image) of the imaging lens102. Likewise, at the photoelectric conversion section 106, a luminousflux is detected that has passed mainly through an area 113 (a secondpupil area), which corresponds to a second direction 111, of theemitting pupil of the imaging lens 102. An A image can be acquired basedon pixel signals from the plurality of photoelectric conversion sections105 and a B image can be acquired based on pixel signals from theplurality of photoelectric conversion sections 106. When the lightquantity distribution of a subject is denoted as f[x, y] and the pointspread function composing the A image of the ranging optical system isdenoted as Pa[x, y], the light quantity distribution A[x, y] of the Aimage can be expressed in a relation of convolution integral as with

$\begin{matrix}{{EQUATION}\mspace{14mu} 1} & \; \\{{A\left\lbrack {x,y} \right\rbrack} = {\sum\limits_{- \infty}^{+ \infty}\;{\sum\limits_{- \infty}^{+ \infty}\;{{f\left\lbrack {{x - p},{y - q}} \right\rbrack}{P_{a}\left\lbrack {p,q} \right\rbrack}}}}} & {{EQUATION}\mspace{14mu} 1}\end{matrix}$

In detection of the distance up to a subject, attention is directed tothe one-dimensional direction of the pair of subject images and theimage gap amount is calculated. In this embodiment, this direction isrendered to be the direction of x-axis. Then, the light quantitydistribution A[x] in the one-dimensional direction of the A image can beexpressed as with EQUATION 2 by use of a line spread function La[x]instead of the point spread function. The line spread function La[x, y]is determined based on the incident angle property of the photoelectricconversion section 105, which detects a luminous flux having passedmainly through the first pupil area, in the pixel 101, and the emittingpupil of the imaging lens 102.

$\begin{matrix}{{A\lbrack x\rbrack} = {\sum\limits_{- \infty}^{+ \infty}\;{{f\left\lbrack {x - p} \right\rbrack}{L_{a}\lbrack p\rbrack}}}} & {{EQUATION}\mspace{14mu} 2}\end{matrix}$

The light quantity distribution B[x] in the one-dimensional direction ofthe B image can also be expressed as with EQUATION 3 by use of a linespread function Lb[x].

$\begin{matrix}{{B\lbrack x\rbrack} = {\sum\limits_{- \infty}^{+ \infty}\;{{f\left\lbrack {x - p} \right\rbrack}{L_{b}\lbrack p\rbrack}}}} & {{EQUATION}\mspace{14mu} 3}\end{matrix}$

If vignetting is caused by the frame of an taking lens, or the like,line spread functions La[x] and Lb[x] forming the A and B images,respectively, become different functions, which causes the A and Bimages to have different shapes with each other. At step S3, thedistance up to a subject is calculated from the pair of A and B imagesby means of a known distance calculating unit. For example, the imagegap amount between the A and B images is acquired by correlationcalculation for the pair of AB images, and the base line length isacquired from the centroid distance between the pair of AB images. Adefocus amount DEF[0] is acquired based on the image gap amount and thebase line length acquired in this manner, and then the distance up tothe subject can be calculated.

At step S4, 1 is added to the processing count N and the process is thencontinued to step S5. At step S5, a provisional defocus amount used inthe following processes is set. In the case when the processing count Nis 1, the defocus amount DEF[0] calculated at step S3 is set, while inthe case when the processing count N is 2 or more, an updated defocusamount DEF[N] calculated at step S8 (described later) is set as aprovisional defocus amount.

At step S6 and step S7, an A′ image (third image) and a B′ image (fourthimage) are formed (created) by modifying the A and B images,respectively. Step S6 is a step in which an image modification filterLa′[x] (the second image modification filter) and the other imagemodification filter Lb′[x] (the first image modification filter) areformed, and includes a process of discretization and a process ofcentroid correction. The image modification filters are formed based onthe image modification functions (the first and second modificationfunctions). The image modification functions are rendered to be, forexample, line spread functions (line image functions) corresponding tothe A and B images, respectively.

FIG. 4 is a flow chart for forming an image modification filter. FIGS.5A to 5D show functions formed at the respective steps.

At step S101, shading of the AB images is estimated based on lensinformation, position information on the imaging surfaces of the rangingpixels having picked up the AB images, and the provisional defocusamount, and then shading correction for adjusting the light quantityratio of the AB images is performed. At step S102, image modificationfunctions are calculated. For example, pupil intensity distributionstored in the recording unit is read out, and the line spread functionLa[x] is calculated based on the lens information, the positioninformation on the imaging surfaces of the ranging pixels, and theprovisional defocus amount set at step S5 (FIG. 5A). At this time, anadjustment is performed as necessary so that the maximum values of La[x]and Lb[x] become identical. This is because the shading correction forthe AB images has been performed at step S101. At step S103, a discretefunction La[x] is formed by discretizing the image modification function(FIG. 5B). This discretization is conducted in accordance with thearrangement spacing of the ranging pixels 101 for a calculation madewith the acquired B image.

The centroid position ga on the x axis of the discrete function La[x] iscalculated by EQUATION 4. i denotes a positive integer, and n denotesthe number of ranging pixels which are used for ranging calculation.

$\begin{matrix}{g_{a} = \frac{\sum\limits_{i = 1}^{n}\;{x_{i}{L_{a}\lbrack i\rbrack}}}{\sum\limits_{i = 1}^{n}\;{L_{a}\lbrack i\rbrack}}} & {{EQUATION}\mspace{14mu} 4}\end{matrix}$

Among the discrete data of the discrete function La[x], the position ofthe discrete data closest to the centroid position ga on the x axis isdenoted as a reference point xga, and the difference between thereference point xga and the centroid position ga is denoted as δx. Atstep S104, the centroid position of the discrete function La[x] iscorrected. Intermediate points between respective data (between samplingpoints) of the discrete function La[x] are interpolated using aone-dimensional function, and the interpolated value at the positionshifted (moved) by δx in the x axis direction from each of positions ofthe respective data is calculated (FIG. 5C). Based on the calculatedinterpolated values, the image modification filter La′[x] (the secondimage modification filter) is formed (FIG. 5D). At this time, anadjustment is performed as necessary so that the maximum values ofLa′[x] and Lb′[x] become identical. The centroid position calculatedbased on the respective data of the image modification filter La′[x] isdenoted as the centroid position ga′. By use of a technique like this,the image modification filter La′[x], which has a shape approximatelyidentical with the discrete function La[x] and in which the centroidposition ga′ and the reference point xga coincide with each other, canbe formed. As to the image modification filter Lb′[x], the imagemodification filter Lb′[x] (the first image modification filter), whichhas a shape approximately identical with the discrete function Lb[x] andin which the centroid position gb′ and the reference point xgb coincidewith each other, also can be formed in a similar procedure.

At step S7, an A′ image and a B′ image having modified image shapes areformed using the A and B images and the image modification filters La′and Lb′. By performing convolution integral on a one-dimensional signalA[x] of the A image with the image modification filter Lb′[x], A′[x] isformed. At this time, the reference point of convolution integral isdenoted as the reference point xgb. The A′ image is expressed byEQUATION 5.

$\begin{matrix}{{A^{\prime}\lbrack x\rbrack} = {{\sum\limits_{i = 1}^{n}\;{{A\left\lbrack {x - i} \right\rbrack}{L_{b}^{\prime}\lbrack i\rbrack}}} = {\sum\limits_{j = 1}^{n}\;{\sum\limits_{i = 1}^{n}\;{{f\left\lbrack {x - j - i} \right\rbrack}{L_{a}\lbrack j\rbrack}{L_{b}^{\prime}\lbrack i\rbrack}}}}}} & {{EQUATION}\mspace{14mu} 5}\end{matrix}$

Likewise, as to the image B, by performing convolution integral on aone-dimensional signal B[x] of the B image with the image modificationfilter La′[x], B′[x] is formed. At this time, the reference point of theconvolution integral is denoted as the reference point xga. The B′ imageis expressed by EQUATION 6.

$\begin{matrix}{{B^{\prime}\lbrack x\rbrack} = {{\sum\;{{B\left\lbrack {x - i} \right\rbrack}{L_{a}^{\prime}\lbrack i\rbrack}}} = {\sum\limits_{j = 1}^{n}\;{\sum\limits_{i = 1}^{n}\;{{f\left\lbrack {x - j - i} \right\rbrack}{L_{a}^{\prime}\lbrack j\rbrack}{L_{b}\lbrack i\rbrack}}}}}} & {{EQUATION}\mspace{14mu} 6}\end{matrix}$

Since La′[x] and La[x] are functions each having an approximatelyidentical shape, and Lb′[x] and Lb[x] are also the same as above, the A′and B′ images acquired based on EQUATIONS 5 and 6 are each rendered tohave approximately the same shape as the original light quantitydistribution f of the subject. At step S8, the defocus amount DEF[N] andthe distance up to the subject are calculated using the A′ and B′ imagesthrough a known method as with step S3. The use of the modified A′ andB′ images causes calculation errors of the image gap amount due to theshape difference between the both images to be decreased and also causescalculation accuracy of the defocus amount to be enhanced. In addition,the use of the image modification filters according to the presentinvention enables to calculate a defocus amount and the distance up to asubject with a higher degree of accuracy than conventional technique.

At step S9, determination on whether or not to repeat the modificationcalculation is given. More specifically, the difference between thedefocus amount DEF[N] acquired at step S8 and the provisional defocusamount set at step S5 is calculated, and is compared with apredetermined threshold value for convergence determination. When thedifference is larger than the threshold value, since the calculateddefocus amount does not converge, the process returns to step S4. Atstep S4, this defocus amount DEF[N] is set as a provisional defocusamount, and re-processing is performed. When the difference hasconverged to the threshold value or less at step S9, the flow iscompleted. The shape of each of the line spread functions La[x] andLb[x] varies depending on the defocus amount. By repeating loopprocessing like this to form an image modification filter based on thedefocus amount DEF[N] close to the right value, the shape error of theimage modification filter becomes small. Therefore, the shape error ofthe modified image is reduced, the accuracy of calculating the image gapamount and the defocus amount is enhanced, and the ranging accuracy isalso enhanced.

By performing ranging using the image modification filters according tothis embodiment along the above flow chart, a defocus amount can becalculated with high accuracy at high speed as compared with aconventional technique. The defocus amount (distance) is calculatedbased on an image gap amount of a pair of images each modified usingimage modification filters. This image gap amount can be acquired byperforming a correlation calculation as one of a pair of modified imagesis shifted, and by calculating the gap amount when the correlation hasbecome highest. The image gap amount calculated at this time isdetermined based on centroid positions and image shapes of therespective images. If the image modification filter has a centroiderror, the centroid position of the modified image changes from thecentroid position of the image at the time before modified. If the shapeof the image modification filter is different from the shape of thediscrete function of the image modification function corresponding tothe right defocus amount, the modified images are caused to havedifferent shapes to each other. These errors bring about an error in acalculated value of the image gap amount, which causes the rangingaccuracy to be degraded.

In this embodiment, the centroid error is reduced by correctingrespective data values of the discrete function to form an imagemodification filter having the centroid position and the reference pointhaving caused to coincide with each other. In addition, the shape erroris controlled by causing intermediate points between discrete data toundergo function interpolation to form an image modification filterbased on the interpolated values. Due to these effects, an imagemodification filter having the small centroid error and the small shapeerror can be acquired, and a highly accurate ranging can be implemented.By using the image modification filter according to this embodiment,convergence in the calculation for ranging through loop processing isenhanced, and ranging can be performed with high accuracy at high speed.The image modification filter according to this embodiment has afunction shape to which the centroid error is reflected. Due torepetition of the calculation for ranging through loop processing, theresult converges to a defocus amount that causes the centroid error andthe shape error to minimize. The image modification filter according tothis embodiment therefore enables to calculate a proper defocus amountat a less loop count as compared to a conventional image modificationfilter and enables ranging to be performed with high accuracy at highspeed.

The reasons why the image modification filter according to thisembodiment can be formed easily at high speed will be described. Thenumber of data of the discrete function used for calculation for rangingis denoted as n, and a number of discrete data is denoted as i. Thevalue of each of data is denoted as Li, and each value of the discretedata is assigned to be L1 to Ln, while L0, Ln+1 and Ln+2 are eachrendered to be 0. Here, the coordinate of discrete data on the imagingsurface is denoted as xi, and the interval is denoted as Δx. Thecentroid position g before centroid correction is executed can becalculated from EQUATION 7.

$\begin{matrix}{g = \frac{\sum\limits_{i = 0}^{n}\;{x_{i}L_{i}{dx}}}{\sum\limits_{i = 0}^{n}\;{L_{i}{dx}}}} & {{EQUATION}\mspace{14mu} 7}\end{matrix}$

The amount of gap between the centroid position g and the referencepoint is denoted as δx. Intermediate points between discrete data areinterpolated with a one-dimensional function, and the interpolated valueat the position shifted by δx from each of the discrete data is denotedas Li′. This Li′ is expressed by EQUATION 8, and the centroid positiong′ is expressed by EQUATION 9.

$\begin{matrix}{\mspace{79mu}{L_{i}^{\prime} = {L_{i} - {\delta\; x\frac{L_{i} - L_{i + 1}}{\Delta\; x}}}}} & {{EQUATION}\mspace{14mu} 8} \\\begin{matrix}{g^{\prime} = \frac{\sum\limits_{i = 0}^{n}\;\left( {x_{i}L_{i}^{\prime}} \right)}{\sum\limits_{i = 0}^{n}\; L_{i}^{\prime}}} \\{= \frac{\sum\limits_{i = 0}^{n}\;\left\lbrack {x_{i}\left( {L_{i} - {\delta\; x\frac{\left( {L_{i} - L_{i + 1}} \right)}{\Delta\; x}}} \right)} \right\rbrack}{\sum\limits_{i = 0}^{n}\;\left\lbrack {L_{i} - {\delta\; x\frac{\left( {L_{i} - L_{i + 1}} \right)}{\Delta\; x}}} \right\rbrack}} \\{= \frac{{\sum\limits_{i = 0}^{n}\;\left( {x_{i}L_{i}} \right)} - {\frac{\delta\; x}{\Delta\; x}{\sum\limits_{i = 0}^{n}\;\left( {\Delta\;{xL}_{i}} \right)}} + {\frac{\delta\; x}{\Delta\; x}\left( {{{- x_{1}}L_{1}} + {x_{n}L_{n + 1}}} \right)}}{{\sum\limits_{i = 0}^{n}\; L_{i}} - {\frac{\delta\; x}{\Delta\; x}\left( {L_{1} - L_{n + 1}} \right)}}} \\{= {\frac{\sum\limits_{i = 0}^{n}\;\left( {x_{i}L_{i}} \right)}{\sum\limits_{i = 0}^{n}\; L_{i}} + \frac{\frac{\delta\; x}{\Delta\; x}{\sum\limits_{i = 0}^{n}\;\left( {\Delta\;{xL}_{i}} \right)}}{\sum\limits_{i = 0}^{n}\; L_{i}}}} \\{= {g - {\delta\; x}}}\end{matrix} & {{EQUATION}\mspace{14mu} 9}\end{matrix}$

As known from EQUATION 9, the centroid position g′ is rendered to be onewhich is shifted by −δx from the original centroid position g expressedby Equation 7, and comes to coincide with the reference point. Asdescribed above, an image modification filter can be easily formed athigh speed according to the technique of this embodiment.

FIG. 6 illustrates the ranging accuracy when modification calculation isperformed using image modification filters according to the presentinvention. The lateral axis in the graph denotes processing count N inthe flow chart of FIG. 2. The vertical axis denotes the error ofcalculated defocus amounts, which are the value obtained by dividing thedifference between the right value of a defocus amount and a calculatedvalue with the right value. The solid line represents the results whenimage modification filters according to the present invention are used,while the broken line represents the results when conventional imagemodification filters La, Lb are used. As shown in the graph, the errordoes not decrease even with increase of processing count N when theconventional image modification filters are used. On the other hand, itis known that, when the image modification filters according to theinvention are used, ranging errors are decreased at less processingcount N and highly accurate high-speed ranging is enabled.

Since the A image and the B image are determined based on the pointspread functions as represented by EQUATION 1, the image modificationfilters may be formed based on the point spread functions instead of theline spread functions to modify the shapes of the images. In this case,the processes at step 6 and step 7 are performed two-dimensionally, sothat the effects similar to the above mentioned effects can be exerted.

The process of forming an image modification filter at step S6 in thisembodiment may be implemented using a different method. For example, atstep S104, centroid correction may be implemented by adding apredetermined value to the respective discrete data of a discretefunction. FIGS. 7A and 7B illustrate the function formed in accordancewith this method at step S104. A predetermined value δL is added to therespective discrete data of a discrete function La[x] (FIG. 7A). Thecentroid position before it has been corrected is denoted as g, thecentroid position after having been corrected is denoted as g′, and thegap amount between the centroid position g and the position of one ofdiscrete data closest thereto is denoted as δx. The centroid position g′after having been corrected can be expressed by EQUATION 10.g′=g+δx  EQUATION 10

The centroid position g′ when the predetermined value δL has been addedto the respective discrete data can be expressed by EQUATION 11. In theEQUATION 11, i denotes a number of discrete data, n denotes the numberof data, xi denotes the coordinate of the respective data, and Lidenotes the respective data values.

$\begin{matrix}\begin{matrix}{g^{\prime} = \frac{\sum\limits_{i = 1}^{n}\;\left\lbrack {x_{i}\left( {L_{i} + {\delta\; L}} \right)} \right\rbrack}{\sum\limits_{i = 1}^{n}\;\left( {L_{i} + {\delta\; L}} \right)}} \\{= \frac{{\sum\limits_{i = 1}^{n}\;\left( {x_{i}L_{i}} \right)} + {\delta\; L{\sum\limits_{i = 1}^{n}\;\left( x_{i} \right)}}}{{\sum\limits_{i = 1}^{n}\;\left( L_{i} \right)} + {n\;\delta\; L}}}\end{matrix} & {{EQUATION}\mspace{14mu} 11}\end{matrix}$

Based on the above Equations 10 and 11, the predetermined value δL canbe expressed by Equation 12.

$\begin{matrix}{{\delta\; L} = \frac{{\sum\limits_{i = 1}^{n}\;\left( {x_{i}L_{i}} \right)} - {\left( {g + {\delta\; x}} \right){\sum\limits_{i = 1}^{n}\;\left( L_{i} \right)}}}{{\left( {g + {\delta\; x}} \right)n} - {\sum\limits_{i = 1}^{n}\;\left( x_{i} \right)}}} & {{EQUATION}\mspace{14mu} 12}\end{matrix}$

Next, an adjustment is performed as necessary so that the maximum valuesof both the image modification filters La′[x] and Lb′[x] are equalized,and the image modification filter La′[x] is formed (FIG. 7B). An imagemodification filter having an approximately identical shape with respectto the original discrete function can be formed by the technique likethis. Then, by calculating the value δL expressed by Equation 12 to addit to the respective discrete data, an image modification filter can beformed at single processing in which the centroid position and theposition of the discrete data are in coincident with each other. Animage modification filter having reduced the shape error and thecentroid error can be easily acquired, and a highly accurate ranging canbe implemented at high speed.

In image modification processes at steps S6 and S7 in this embodiment,the inverse function of a line spread function may be used as the imagemodification function. The one-dimensional signal of the A image isdenoted as A[x] and the image modification filter formed using theinverse function of the line spread function corresponding to the Aimage (first image) is denoted as La′−1[x]. By performing a similarprocess as with step S6, the image modification filter La′−1[x] (thefirst image modification filter), in which the centroid position and thereference point are in coincident with each other, can be formed. Theimage shape is modified by performing convolution integral on aone-dimensional signal A[x] of the A image with the image modificationfilter La′−1[x]. At this time, the reference point of the convolutionintegral is rendered to be the reference point xga of the imagemodification filter La′−1[x]. The A′ image, the shape of which ismodified from that of the A image, is expressed by EQUATION 13.

$\begin{matrix}\begin{matrix}{{A^{\prime}\lbrack x\rbrack} = {\sum\limits_{i = 1}^{n}\;{{A\left\lbrack {x - i} \right\rbrack}{L_{a}^{\prime - 1}\lbrack i\rbrack}}}} \\{= {\sum\limits_{j = 1}^{n}\;{\sum\limits_{i = 1}^{n}\;{{f\left\lbrack {x - j - i} \right\rbrack}{L_{a}\lbrack j\rbrack}{L_{a}^{\prime - 1}\lbrack i\rbrack}}}}}\end{matrix} & {{EQUATION}\mspace{14mu} 13}\end{matrix}$

Likewise, as to the B image, the image shape is modified by performingconvolution integral on a one-dimensional signal B[x] of the B imagewith the image modification filter Lb′−1[x] (the second imagemodification filter), which has been formed using the inverse functionof the line spread function corresponding to the B image (the secondimage). The B′[x], the shape of which is modified from that of the Bimage, is expressed by EQUATION 14. At this time, the reference point ofthe convolution integral is rendered to be the reference point xgb ofthe image modification filter Lb′−1[x].

$\begin{matrix}\begin{matrix}{{B^{\prime}\lbrack x\rbrack} = {\sum\limits_{i = 1}^{n}\;{{B\left\lbrack {x - i} \right\rbrack}{L_{b}^{\prime - 1}\lbrack i\rbrack}}}} \\{= {\sum\limits_{j = 1}^{n}\;{\sum\limits_{i = 1}^{n}\;{{f\left\lbrack {x - j - i} \right\rbrack}{L_{b}\lbrack j\rbrack}{L_{b}^{\prime - 1}\lbrack i\rbrack}}}}}\end{matrix} & {{EQUATION}\mspace{14mu} 14}\end{matrix}$

Since La′[x] and La[x] are functions each having an approximatelyidentical shape, and Lb′[x] and Lb[x] are also the same as above, the A′and B′ images acquired based on Equations 13 and 14 are each rendered tohave approximately the same shape as the original light quantitydistribution f of the subject. A blur of an image of the subject due todefocus can be eliminated and the distinct A′ and B′ images can beacquired by implementing the process like this. The defocus amountDEF[N] and the distance up to the subject are calculated using the A′and B′ images through a known method and, thereby, a highly accurateranging can be implemented. Incidentally, in this embodiment, althoughthe method is presented by which the determination whether or not toperform the calculation again at step S9 is made based on theconvergence state of the defocus amount (distance information on thesubject), a different method may be employed. The determination may alsobe made based on the convergence state of, for example, the shape of theimage modification filter, the base line length, the image gap amountbetween the AB images, or the like. The determination may further bemade by counting the number of processing times with respect to apredetermined number N having set in advance.

In this embodiment, a process for estimating the magnitude of the imagegap amount between the first and second images may be provided, andranging may be performed using image modification filters according tothe present invention when the image gap amount is small. FIG. 8 is aflow chart in this case. The image gap amount between the AB imagesdecreases as the defocus amount becomes small. On the other hand, acentroid error amount and the error amount of image gap resultingtherefrom are not proportional to the defocus amount. For this reason,the smaller the defocus amount, the larger the influence of the error inthe image gap amount resulting from a centroid error becomes, andranging accuracy becomes worse. Further, convergence in a loop processalso becomes worse and the ranging time increases. As shown in FIG. 8,step S10 for estimating the magnitude of the image gap amount betweenthe AB images is therefore provided. When the image gap amount is small,the process is advanced to step S3 onward, and ranging is performedusing the image modification filters according to the present invention.When the image gap amount is large, since the influence of the centroiderror is small, the process is advanced to step S11, and ranging isperformed using a conventional method. For example, ranging is performedusing conventional image modification filters each having a centroiderror, or ranging may be performed using acquired AB images withoutexecuting the process of image modification. At step S10, the criterionof estimating the magnitude of an image gap amount can be determined bycomparing the error in the image gap amount resulting from a centroiderror with the tolerance of the image gap amount. The tolerance of theimage gap amount is determined based on an aimed ranging accuracy andthe configuration of the ranging apparatus. By performing the processalong a flow chart like this, an appropriate ranging can be performeddepending on an approximate distance up to a subject (defocus amount),and a highly accurate ranging can be implemented at a higher speed.

The results of ranging by use of the ranging apparatus 100 according tothis embodiment can be used for, for example, focus detection of imagingoptical systems. A ranging apparatus 100 according to this embodimentenables to measure the distance up to a subject with high accuracy athigh speed, and the gap amount between the subject and the focalposition of an imaging optical system can be known. By controlling thefocal position of an imaging optical system, the subject can be focusedwith high accuracy at high speed.

As with pixels 101 in the ranging apparatus 100 according to thisembodiment, by arranging plural photoelectric conversion sections in asingle pixel, image signals of the pixels 101 can be created using thesignals acquired by the photoelectric conversion sections 105 and 106.If pixels like these are arranged in all the pixels in the imagingdevice 103, image signals can be acquired at each of the pixels, and animage of the subject can be acquired together with performing ranging.Moreover, by performing ranging using pixel signals extracted from anarbitrary group of pixels in the imaging device 103, the distance up toan arbitrary imaging area (subject) can be measured. By extracting pixelsignals from respective areas in the imaging device 103 and byperforming ranging using them, a range image can also be acquired.

The ranging apparatus 100 according to the present invention is notlimited to the structure of this embodiment. It may be allowed todispose the pixels 101 in a portion of the imaging device (solid-stateimaging device) 103, which is configured by arranging pixels in a plane,and to dispose pixels for acquiring an image in the other portionthereof. Ranging is therefore performed using the partial pixels 101,while acquisition of an image of the subject can be performed using theremaining pixels. The imaging device 103 can be configured so thatpixels are disposed in one direction on the imaging surface and rangingis performed by detecting the image gap amount in the one direction. Theshading members 108, 109 and the photoelectric conversion sections 105,106 disposed in the respective pixels 101 may be arranged in ydirection, and ranging may be performed using signals acquired by therespective photoelectric conversion sections. With the structure likethis, ranging of a subject having a variation of contrast in the ydirection can be performed. Another structure may also be possible inwhich pixels each having the shading members 108, 109 and thephotoelectric conversion sections 105, 106 arranged in x direction andalso in y direction are mixed, or in which the pixels are arranged in adiagonal direction (xy direction). Ranging can be performed withappropriately choosing signals used for ranging depending on thedirection in which the contrast of the subject varies.

A structure may also be possible in which plural pairs of pixels 120 and121 illustrated in FIGS. 9A and 9B are arranged. The pixel 120 isprovided with a micro lens 107 and a shading member 122 on a board 104and an photoelectric conversion section 105 mounted in the board 104,and the pixel 121 is provided with a micro lens 107 and a shading member123 on the board 104 and an photoelectric conversion section 106 mountedin another board 104. The pixel 120 is able to receive light incidentfrom a first direction, while the pixel 121 is able to receive lightincident from a second direction. AB images can be acquired based onsignals acquired by the photoelectric conversion sections 105, 106 inthe pixels 120, 121, respectively. Since the spacings between theshading members and between the photoelectric conversion sections becomelarge, respective pixels can be easily fabricated.

Further, each of pixels included in a ranging apparatus 100 may bestructured using a waveguide illustrated in FIG. 10. A pixel 130 has awaveguide 131 arranged on the light-entering side (+z side) of a board104 and photoelectric conversion sections 105, 106 disposed in the board104. The waveguide 131 includes a core portion 132 and a clad portion133. The core portion 1322 and the clad portion 133 are formed of amaterial being transmissive in the imaging wavelength band, and the coreportion 132 is formed of a material having a higher refractive index ascompared with that of the clad portion 133. Therefore, light can beconfined within the core portion 132 and the clad portion 133 and lightcan be allowed to propagate therethrough. A luminous flux having enteredthe pixel 130 from the outside propagates through the waveguide 131 andis emitted into the board 104. The luminous flux 110 having entered thepixel 130 from the first direction propagates through the waveguide 131and is able to be introduced to the photoelectric conversion section105. On the other hand, the luminous flux 111 having entered the pixel130 from the second direction propagates through the waveguide 131 andis able to be introduced to the photoelectric conversion section 106. Byconfiguring the device in this manner, light depending on the incidentdirection thereof can be effectively detected.

The structure of a backside incident type may also be possible in whicha waveguide constituted by a core portion and a clad portion is providedin a board 104. By configuring the device in this manner, light enteringfrom the backside of the board (light propagating in +z direction) canbe detected. Wiring and the like can be arranged on the front side ofthe board 104, by which the propagation of incident light is preventedfrom being interfered by the wiring and the like. In addition,space-wise restriction due to the wiring and the like is lessened, andthe incident light can therefore be effectively introduced tophotoelectric conversion sections. The ranging apparatuses and theranging methods according to the above embodiments are able to befavorably applied to an imaging optical system such as a digital stillcamera, a digital video camera, or the like.

According to the present invention, a ranging apparatus, a rangingmethod and an imaging system which enable to perform highly accuratehigh-speed ranging can be actualized.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2012-095648 filed on Apr. 19, 2012, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. A ranging apparatus comprising: an imagingoptical system arranged to form an image of a subject; an imaging unitincluding a pixel group arranged to acquire a first image and a secondimage, the first and second images being formed by luminous fluxeshaving passed mainly through a first pupil area and a second pupil area,respectively, of an emitting pupil of the imaging optical system; and acalculation unit, wherein the calculation unit is configured to create athird image by performing convolution integral on the first image with acorrected first image modification function as well as to create afourth image by performing convolution integral on the second image witha corrected second image modification function, and to calculate adistance up to the subject by comparison of the third and fourth images,wherein the corrected first and second image modification functions areformed by causing respective centroid positions, which are calculatedbased on data of sampling points of respective first and second imagemodification functions, to each coincide with the sampling point closestto the centroid position, the sampling points being corresponding topixel arrangement of the pixel group, and wherein the convolutionintegral is performed by taking the sampling point closest to thecentroid position as a reference point.
 2. The ranging apparatusaccording to claim 1, wherein the corrected first and second imagemodification functions are acquired by interpolating between data of thesampling points of the respective first and second image modificationfunctions with a one-dimensional function, and by using the valuesinterpolated with one-dimensional function at the positions shifted fromthe respective sampling points by a difference between the centroidposition and the reference point.
 3. The ranging apparatus according toclaim 1, wherein the corrected first and second image modificationfunctions are acquired by adding predetermined values expressed with afollowing equation to data of the sampling points of the respectivefirst and second image modification functions:${\delta\; L} = \frac{{\sum\limits_{i = 1}^{n}\;\left( {x_{i}L_{i}} \right)} - {\left( {g + {\delta\; x}} \right){\sum\limits_{i = 1}^{n}\;\left( L_{i} \right)}}}{{\left( {g + {\delta\; x}} \right)n} - {\sum\limits_{i = 1}^{n}\;\left( x_{i} \right)}}$where, δL: Predetermined value i: A number corresponding to samplingpoints of image modification function n: The number of sampling pointsused for ranging xi: Coordinate of sampling points Li: Data of samplingpoints of image modification function g: Centroid position calculatedbased on data of sampling points of image modification function δx: Gapamount between centroid position g and reference point.
 4. The rangingapparatus according to claim 1, wherein the calculation unit isconfigured to calculate a distance up to the subject by comparison ofthe first and second images, and to form the corrected first and secondimage modification functions based on the calculated distanceinformation.
 5. The ranging apparatus according to claim 1, wherein thefirst image modification function is a line spread functioncorresponding to the second image, and the second image modificationfunction is a line spread function corresponding to the first image. 6.The ranging apparatus according to claim 1, wherein the first imagemodification function is an inverse function of the line spread functioncorresponding to the first image, and the second image modificationfunction is an inverse function of the line spread functioncorresponding to the second image.
 7. The ranging apparatus according toclaim 1, wherein the pixel group includes a partial pixel group of asolid-state imaging device configured by arranging pixels in a plane. 8.An imaging system arranged to perform focus detection of the imagingoptical system based on ranging results of the ranging apparatusaccording to claim
 1. 9. An imaging system comprising the rangingapparatus according to claim 1, the imaging system arranged to acquire adistance image.
 10. A ranging method of measuring a distance up to asubject utilizing an imaging optical system arranged to form an image ofthe subject to acquire a first image and a second image using an imagingunit including a pixel group, the first and second images being formedby luminous fluxes having passed mainly through a first pupil area and asecond pupil area, respectively, of an emitting pupil of the imagingoptical system, the distance up to the subject being measured based onthe first and second images acquired by the imaging unit, the methodcomprising: forming a first and second image modification filters;creating a third image by performing convolution integral on the firstimage with the first image modification filter, and also creating afourth image by performing convolution integral on the second image withthe second image modification filter; and calculating a distance up tothe subject based on an image gap amount between the third and fourthimages, wherein the forming first and second image modification filtersincludes discretizing and correcting, and wherein the image modificationfilters are each formed through discretizing an image modificationfunction according to a pixel arrangement of the pixel group to form adiscrete function, and, correcting a centroid by taking a position ofthe discrete data closest to a centroid position of the discretefunction from among a discrete data of the discrete function as areference point to cause the centroid position to coincide with thereference point.
 11. The ranging method according to claim 10, whereinthe correcting centroid includes interpolating between discrete data ofthe discrete function with a one-dimensional function, and treatinginterpolated values with the one-dimensional function as respective datavalues of each of the image modification filters, the interpolatedvalues being at positions shifted from respective discretized points bythe difference between the centroid position and the reference point.12. The ranging method according to claim 10, wherein the correctingcentroid includes treating values acquired by adding predeterminedvalues expressed with a following equation to discrete data values ofthe discrete function as respective data values of each of the imagemodification filters:${\delta\; L} = \frac{{\sum\limits_{i = 1}^{n}\;\left( {x_{i}L_{i}} \right)} - {\left( {g + {\delta\; x}} \right){\sum\limits_{i = 1}^{n}\;\left( L_{i} \right)}}}{{\left( {g + {\delta\; x}} \right)n} - {\sum\limits_{i = 1}^{n}\;\left( x_{i} \right)}}$where, δL: Predetermined value i: A number corresponding to discretedata included in discrete function n: The number of discrete data usedfor ranging xi: Coordinate of discrete data Li: Value of discrete datag: Centroid position of discrete function δx: Gap amount betweencentroid position g and reference point.
 13. The ranging methodaccording to claim 10, wherein, the first and second image modificationfilters are formed based on information on a distance up to the subject,the distance being measured by comparison of the first and secondimages.
 14. The ranging method according to claim 10, wherein an imagemodification function used to form the first image modification filteris rendered to be a line spread function corresponding to the secondimage, and an image modification function used to form the second imagemodification filter is rendered to be a line spread functioncorresponding to the first image.
 15. The ranging method according toclaim 10, wherein an image modification function used to form the firstimage modification filter is rendered to be an inverse function of theline spread function corresponding to the first image, and an imagemodification function used to form the second image modification filteris rendered to be an inverse function of the line spread functioncorresponding to the second image.
 16. The ranging method according toclaim 10, further comprising estimating whether to perform the creatingthird and fourth images again based on information on the distancemeasured by comparison of the third and fourth images, wherein the firstand second image modification filters are formed based on theinformation on the distance.
 17. The ranging method according to claim10, further comprising estimating the magnitude of an image gap amountbetween the first and second images, wherein, when the image gap amountis estimated to be small at the estimating magnitude, the distance up tothe subject is calculated through the creating third and fourth imagesand the calculating distance.